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ABSTRArr 

Th« Michiqan 8tat# Univeraity protocol iitat<>rial8 on 
Irtrninq w«r« ficld-tvatad at th« Dnlveraity of South Florida 
(U.S. P.). Th« f i*ld-t«atinq at U.S. P. did not occur und^r controll«d, 
•xp«rin«r)tal eonditiona. Th« inforiMtion n4*da of the protocol 
i«v«lop«ra w«r« balanced with th« instructional oblidatlona of th« 
instructors and the sducational n««ds of th<» students in the claeaea. 
Instructional iNsna<ienent problens ani error sources were hiqhlidhted. 
error sources ineludsd operant learning^ positive reinforceaient» 
neqative reinf orceewnt, shaping, respondent learning, and model 
learning. (Appendixes of field«test naterial are included.) (MJM) 



ERIC 



riLMCD PROM BEST AVAILABLE COPY 



o 



IJii I VI' r s i t )• of StMilh I'li'iiilii P i ( 1 il Test 

of 

Tlir Hiiliii;riii St;iti' Dnivtrsity I'rotcicol Mati-ritils on l.i'ii ni i iij; 



(Iriiiil NtmbcT S.n.l:. 720-or.O 



I'innl Keporl 



By 



J ni)C s C . Di c I. i Ti s on 
Fi o ] il Test P i J cct or 



Dci>;irtnciil of Apjilicd Lil u c :» t i onn ) Sciences 
Col 1 V j;C of i (hiCiit i on 
^ Dn i VL' r s i t y of Sou M) V 1 or i ua 

^s. Tanpii, Fieri ila 33620 



00 MOT ffWCIMAIIIiV 

CAnonPOtmonon^oucv 



TAELi; 01' CONThNTS 



Objectives 1 

Procedures 1 

Instructional Unnoccricnt Problerius 4 

Error Sourcer. 6 

Operant Icrninp 7 

Positive He . nf orccr.cnt 9 

Negative Ucinl orccmcnt 10 

Shapinc 11 

Respondent Learning 12 

Model Leai'innfi \A 

APPIiNDIX 

Figure 1 Student Analysis Form 17 

Table I f umber of Hrrors on Kach Concept Test Txpresscd as 

Percent of Total Hrrors Made on the Six Concept 

Tests 19 

Tabic II Number, Percent of All Students Achieving Figlity 

Percent Accuracy on I'oncept Cjiterion 'I'ests 20 

Table 111 Nui^ber, Percent of Attribute Frrors on Operant 

Lcarninp, Criterion Tost 21 

Table IV Student Lvalu.ition of Operant Cond i t i on i r p; Unit .... 22 
Table V Number, Percent of Attribute trrors on Positive 

Reinforcement Criterion Test 23 

Table VI Student P.valuation of Positive Rcinforccnent Unit .. 2A 
Table VII Number, Percent of Attribute Errors on Negative 

Reinforcement Criterion Test 25 

Tabic VIII Student Evaluation of Negative Rcinforccnent Unit.. 26 
Table IX Number, Percent of Attribute Errors on Shaping 

Criterion Test 27 

Table X Student Pvaluation of Shaping Unit 28 

Tabic XI Number, Percent of Attribute Errors on Respondent 

Learning Criterion Test 29 

Table XII Student Evaluation of Respondent Learning Unit .... 30 
Table XIII Number, Percent of Attribute Errors on Model 

Learning Criterion Test 31 

Table XIV Student Evaluation of Model Learning Unit 32 



EKLC 



.1- 



Ob i e c 1 i v v^r^ 

The Michigan State University (M.S.l)») ])roto(ol natorials on 
Icnrninn were field tcstdl at the University of South l'](»r^da (U.S. P. 
with finruicial snnj^ort froM the State of Tlorida l^rotocols 'Mojcct, 
Field testinR the M.S.U. pre* ocoj materials was undtrta):en at U.S.J-, 
to obtain data related to several najor concerns: 

1. Khat j^)'ol)lems occur in the i.ianaf.eia'nt of instruction 
wh en a ]> r o t oc o 1 s p:i c huge 3 s incorporated in an 

c s t n b 1 i li d c o u r s e ? 

2. Khat arc the surctss rates of students at different 
institutions wliicli urc tl)e same pmtoco] ma t e r i :j 1 s ? 
Succesf. rate refers simply to tlie percent or pro]>ortio,i 
of stud en Is acliievin^". a pr e - spc c i f i e d level of accuracy 
on the c v i t e r i o n t c .s t s . 

3. U'hat aspects of tlie ])rotoco] packare seem to be sources 

of confusion and error in student Mastery of the concepts? 

A, Vfhrit are the instructional outcojnes wliicli seen to be 
unexpected cons ecjuen ces of usinf, protocol ir.a teri 1 1 s? 

V TO c e d u r c 

Aliijost a year before the field testinj; was conducted at USr , 
l'rofcsv.ors Judy Henderson, Joe Byers, and Bruce Burke of MSU i::et with 
several members of the USl- educational psychology depart:. ^nt to 
discuss the development of their protocol materials on huj/.an learninr. 
Fol 1 ov. i n that i n t rodtict i on and before introducing the protocol 
nateriuls in the classes at USI', F^rofessors Pickinsun and Kong 
visited MSU to observe the instructional practices used in the 
bepinnin^; educational psycboloj^y course and to discuss their protocol 
development staff's needs for data of certain types coming from 
field testing. 

Acquisition of the protocol ri.iti rials, rcrrui ti:ient USH 



•2- 

ciluc a t i oi) ;» 1 psytlioloj;)' faculty to p;n't i ( i jmt c , .'tk! XUv r i cmi t ;i t i on 
of the volunteer faculty ben^in in the fall (Qu.irter 1) of 1971. I.acli 
fnculty partic3j)nnt received a cojripletv set of i»rotocoI natcrinls v.^ll 
ill adv4iiue of the^tiiic that tlie luaterials were \o he introducetl in his 
or her clr.^s. The protocol lilms were pi-eviewed and discu.sed. A 
su1»sequ(nt visit I))- Trofessor Burke provided t lie i'aculty anotlier 
opportunity to raise i[uestics and receive tlic l^cnefit of tlie MSU 
cxj)cr i crice , 

Duplication and asseiihly of the printed r.ioVerialr, a costly and 
iiwhward proce:^s undcj' the USl conditiojis, were completed f .rly in 
Quarter 11. Di s s c nn n a t i on of the instructional i:;aterials was handled 
by ercli instructor according; to l»is or lier judf^ericnt of the most 
suitable procedure. In. most instances, instruct oi*s handed out the 
appropriate materials in the Meeting p^ior to the class luetin^ wlun 
the paiticular concept would he studied. In one instance students 
received all the printed materials at one t i lae along with a calend;ir 
showing concept testing dates ami concept: filr.i projccti 'Wi dates anil 
t i ries . 

Be for? distributing the materials in class, students were t (> 1 d 
that they would be using instructional materials which were being 
developed to facilitate the learning of key concepts in human lerrning. 
It was em]>hnsi2ed that the final form of the instructional materials-- 
both printed and f i 1 iicd- -woul d be dcternuned in part by the types of 
exi>criences the students had using them. The instructors then 
answered only questions regarding the materials and tlie i)roccdurcs. 
The students* protocol performances were included in every case in 
the computation of the students' final grades for tlie course. 
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In all but two instances, the protocol nintoriiils were iistd as 
part of the in-class activity. Students read the prc-filin materials 
before coming to class, and in cla?s tliey viewed the filns and worlcd 
throu}:li ihc film ^uidc. The instructor answered questions, noted 
sources of confusion, nade judfjiicnts about the authors' intended 
mcanin^is and t^'i^^'^^^l) i^cted to hclj) individual students as they 
experienced difficulties. Small grouj) discussions frecjuently occur) ed, 
but were not designed by the instructor as part of the i)rocedure. 
At the end of the class period student:, took t!ic concept iiuistery 
test. 

In the tv;o exceptional instances, instructors designed tlieir 
courses so that students devoted more time to out-of-clnss study and 
projects. The i)rot oco 1 f i 1 ms were tclccas. on c 1 os c d - c 1 rcu i t to the 
IJSF Learning; Center where students could view the filns in privnle 
carrels us many times as the broadcast tine peniiittcd. The telecast 
schedules and the out-of-class testing arrangcLicn t s were provided 
to the students witli the printed protocol materials. 

In every case instructors added the pr o t oco 1 . ck a ;ic to their 
regular instructional routine witliout deleting other instructional 
objectives. Without exception, therefore, the learning load on 
students was heiglitoned by the use of the protocol materials. 

The fi eld- test ing conducted at USF clearly did not occur under 
controlled, experimental conditions. The director of the USF field 
tost sou<»ht to balance the information needs of the protocol developers 
with the instructional obligations of the instructors and the educa- 
tional nncds of the students in the classes. The data are, therefore, 
sometimes quite ambiguous and perhaps, never r.orc than merely 
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5Uj:jicstivc of answers to the i:;ajor concerns as thvy apply to the 
USI* cxpcri encc . 

1 nsl riic t 1 on :i ] Mann n rn c n t Prol i l /'n is 

Apart fi'oni the printing; and assoirbling problems in prcj>nrinf, 
the ])rotoco] materials for use at USP, the distribution, tlie sub- 
sequent collection and stoj-a^^e of the materials for later re-use 
posed obvious lof;istic;il probleir.s not ordinarily found in college 
classroom iiwi n a guDen t . These i:>roblems seenj to be unique to the 
activity of f i e 1 d - t e s t i n^. and are reported, not as limitations of 
protocol niatirlals, but as a Managerial probleji in field testing. 

One major soui'ce of difficulty whicli plap.ued all instructors 
was the film i)rojection. Macliine failures, operiitor errors, sound- 
track problems, film breakage, and daina^,o to projcctois occurred 
during tlie field test. The sliort supply of trained film projectionists 
from IJSP's Instructional Services forced the par t i c i pat i n instructors 
to show tlio films tliemselves. Much of tlic difficulty experienced 
seemed to be the result of operator ignorance and tlie reliability 
of the equ i ]>ment . 

A secoiul major source of difficulty was the problem of test 
security. VJith multiple class sections using the protocol materials 
and with their use in consecutive acadeiiic terms, tlie possibility 
of losing test security seemed real. At the same time, students 
wanted feedback about their performance which meant review of 
questions and answers. The policy was to let individual students 
examine theii tests, cliallengc the answer key and then sufficiently 
question thcti to teacli the concept attribute on which the error was 
made. Students were not allowed to use paper and pencils when re- 
gj^l^" viewing the materials. 



A tliird major na n o ci i c n t prol'lr;i wrosc in us in;; the final 
protocol cxaiiiina t i on . Student ju* r f o n.mn c c was surprisingly bad-- 
Icss than 20 percent achieved 60 percent aceura cy- - and in lij:lit 
of their re 1 a t i ve • s uc ce s s on tlie individual conccjU test, tlic 
students' results were cni>rji:ousl y disappointing: to theii and to 
their instructors. Class morale in each instance was sc) .severely 
shaKcn tliat all of the instructors quit usinj: tlic final exaninalion. 
Those instructors \;ho wanted a final cNuinination over the j)rotoco]s 
uscul tlie J) re -test foriu, 

A less seriour, inanaf.e],:unl i)roh]ci,i occur jcd often cnou^ih tliat 
it warrants nen t i on i n,^; . When for exar.tplc, students disaf,reed v.itli 
tl)C answers to tlie case- study questionr. in t Ik^ f i 1 ni ^^nd tlieir 

instructors also judged the JlJli^ljcJ-s ansv.er in error, it i)roduced 
amonj; soijc students a skepticism about tlic concej^ts, the materials 
and the objectives of the protocol unit. On the other hand, many 
students could accej^t the idea that d i sagrcer:en t occur bet\;een 
or amonj; professionals without using the d i s a r ccnien t s as a basis 
for disnissini; the materials as a useful source of import ant 
inforna t i on . 

Instructors reported that many (at times about 7h'o of a class) 
students in their haste to get to the test-like examples at the end 
of the film guide failed to read carefully the pre-filu units and 
the film guide's explanatory materials. The behavior of students 
seemed to be oriented toward the successful completion of tlie concept 
tests and, therefore, they seemed to prefer spending their time on 
test- like questions even when their !;:a.stcry of the concept's attri- 
butes fell short of the requisite comjutcnce level. 



Hrror Sources 

The critcrioi^ tests for the six concepts v;ere of the short -an sv. or , 
essay variety. Te^t scoring reliability seciis to have been a bit of 
; problem, at least when instructors conparcd notes on their scorin;^ 
of selected items. Lfforts were nado to reduce bctv.cen- 1 eacher 
variability in scoring;, but no r^u an t i t a t i ve data were obtained to 
d e 111 o n s t r a t e the effect. 

The criterion test construction warrants passing cor-ment. 
Efforts v;ere needed to balance tlie scoring weiglits for the several 
parts of a ci^^-ri question. For example, identifying tlie item as 
representative or non - r epre sen t a t i vc of the concept was worth one 
point. By sliowing which attributes were present to make tlie itcD 
a representative of the concept students could earn the nuinbcr of 
points equal to the nuiaber of attributes. However, a non-example 
always yielded points in the ariOunt of tlic nunbor of missing 
attributes. In almost every case non-examples were characterized 
by one riiissing attribute and therefore v.erc worth one point. 

Before exar.ining tlie separate concepts to locate sources of 
disproportionate error, it is useful to note that the variation 
in the percent of students achieving tlie 80 percent accuracy criterion 
on the six concept was not ctppt^-rent ly a strong function of the number 
of attributes in the concept. As shown in Table I, the lowest 
percent achieving tlie jn* escribed i> erf or nance criterion le\'cl was 
in Operant Learning, a concept with four attributes while the 
highest was in Model Learning with five attributes. The mean 
accuracy rate of the two three-attribute concepts (Shaping and 
Negative Reinforcement) was 73 percent; for the two four-attribute 
^ concepts (Positive Reinforcement and Operant Learning), S9 percent; 



and for the two f i v c - a 1 1 r i bu t c concepts, (Uc;;pondcnt Learning and 
Model Learning), 66.2 percent. 

The variation in concept difficulty as reflected in the percent 
of all errors occurri on a given concept test is shown in Table IK 
Agair, the relationship between nunber of attributes and tJie percent 
of errors r^ccned to be a positive one. The average percent of all 
errors occurrin;^on the th ree - a 1 1 r i but e conccptr was 7.8; on the 
four attribute concepts, 18,7; on the five attribute cone ep t s , 23 . 5 *I . 

Concept difficulty is partly det eri.niiied by the nunlur of attribute 
but the different error rates could as well be a functicii of the 
greater nuv.ibcr of responses denumded on tlie criterion tests for the 
more complc^x concepts. Until the crif^rion tests are balanced r.o 
that the nuinber of responses are held constant across the concepts, 
no firm conclusion can be reached about the sources of variiition in 
error rate . 

Finally, tliere appeared to be a 1 e arn ing -how- t o - 1 earn phenonenon 
associated with perfornance on the protocol concepts. The low 
achievement rate in the Opcrj^nt Learning concept, the first one in 
the protocol package, seemed to bo a function of its no\;ncss to the 
students as an instructional procedure. After the Operant Learning 
concept in all of the class sections, student performance was 
substantially better on the remaining concepts with the possible 
exception of the Respondent Learning concept. 

Oporan t Le timing (OL) As shown in Tablelll, almost forty percent of 
the errors on the OL criterion test were made on the Tenporn 1 Relation 
ship attribute. The remaining errors were distributed about equally 
among the other three attributes. Hxamination of tJie written text 



explicit inn Tc iiPO r ; i 1 Rc 1 at i on sh i ]^ sliows tlut llic cr]>hasis i: on 

the 3 !:^r: e(] i a t c appearance of a conr,cquence fol lowing, the l)el»;ivior. 
The labeling: of the attril)Ute nay be a source of confusion sinply 
because it i? f;,oncral ratlier than specific one, Teiaporal re- 
lationsliips exist, between or anonj', any I wo or more events, and 
til ere is notliinj, about tlie expression to s\if,j',est casual connection, 
only relative positions in the passage of t i mc , 

The concept lui f.ht liavc pioduced fewer errors liad tlic t wo 
attributes-- C^ij^ e([n cnces a n d Te n ^^o ra 1 K_M a t i on sh i r s - - b e en c oii:b i n e d 
to ))riu]uce tlie following attribute: iir.medvate consetiucnce , On 
paf.e 30, botto)!! iKiraj;raph, consetiuence is defined as . , . any event 
that i iiiMcd i at e 1 y follows a behavior.^ If conscqueiu'.e ii::plics 
i m me d i a t e o u t c o in e , t h e Te npo ra 1 Re ] a t i on s h lp_ i s a s ii j) e r f 1 u o u *> 
a 1 1 r j b 11 1 o . 

The attribute of })c lia\' i or Ch a n r,e recpiircd supj^l cMcnta 1 dircuscion 
in class l>ccause nany students had not concept ua 1 1 red behavior as 
response frequc^ncy. In tliis case, the absence of prerequisite 
entcrijip, beliavior dcl;:ycd tlic acquisition of tlie concc])t even when 
presented in the protocol foriiiat . 

0 . li * ]• i 1 > As si.uvn in Tal)lc IV, filu details as judp,ed by 

students were of avcrafc or satisfactory quality. Sound quality 

was below expectations as words were garbled and background noise 

was distracting to listeners. According to student rosi)ondents 

the film content was quite weil integrated with the written materials. 

On page A \ of the OL f i 1 :n guide, the tcaclier is described as 
having ''rewarded those who had lined up approju* i a l e 1 y . in fact, 
the teacher in the film rewarded those students who said they had 
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hchavcd appropriately and did not rcwnrd the student who said lie had 
not behnvcd in line. In effect, tlic teaclicr was rcinforcinj: verbal 
expressions of compliance. 

Posit i V c j^c; i n f u y com o n (PR) Oata reported in Table V show tliat More than 
sixty percent of tJie errors on tlic V\\ test occurred on two attributes-- 
i:on t i n; ency and Terij >o r;. 1 l^^illiil J_I^Ji^J»_Ul. Tlic notion of contin^.ency is 
presented on page 58 as ''The revardin^; of stiriulus must be a direct 
consetiucnce of tlic bcliavior tliat was err. i 1 1 ed - - i . e . , tlie rewarding; 
stimulus is witlihcld unless the behavior is enitted.'* 1\\q ]>re ju)s i t i on 
of *'»t the bci^inning was a nuisance source of coT^fusion, repe;ited throupli- 
out the cha})ter. More inportant frui.i the fielci test iacult)s' view- 
point, was tlie absence of the word depc ncv. t. A contingent event is 
an event dependent upon an antecedent occurrence. 

Picld test faculty and students also were confused by the al^sence 
of the l>e)Ki V i or Chan^.e attribute wliicli was present in tlic O.L. conccj/t 
presentation. Te elm i ca 1 1 y , positive re i n iorceren t can only be inferred 
fronj a set ol' conditions (events) wliich include a behavior, subsequent 
presentation of a stir.uilns event, and then an i nrreas o in the behiiv-or's 
frequency of occurrence. While the absent attribute could not be shown 
directly as a source of difficulty, it presented a departure frcia the 
explicit and complete iriodel of behavior chan^;c presented in the O.L. 
chapter and as an inconsistency was a r-ourcc of 10{;ica] difficulty in 
the PK section. This particular difficulty can also be described as a 
definitional problem. In the O.L. discussion, a functional definition 
was used; in the PR, a procedural definition. On this point the field- 
test faculty strongly emphasised the desirability of tcacliers relyin}; 
on tlie functional definition while pointing out the problens which 
f ol 1 ow f roiTi us i nfi .proeedura 1 dc f i ni t i on s . 
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f In . Student reaction to the TR filn reported In Tallc VI 
iraft that the visual repreac**t at : on was rcasonjiMy t^^^ • th«t the 
apund wa> audible and clear and that the fils vb% veil intrj;rateJ 
vith the written «^ater:alf. The field-te»t ataff i^aa aatUfied 
with the filn at a represent at 1 on of po5^itivc rc ir.^orcer.tnt * 

HegJ»t I ve l ^r i n forcenrnt (KR) The data in Table VII reveal that 
alaoft half of the errora iicre aade un the tehi> v i *>r attribute. 
The KR 0ateriala» fron the field-teat staff*a viewpoint, led to 
confusion utout the ce^sninc of behavior because it Is cc>n!.t «.t.t Ijr 
inplled thro phout the three attributrs. f res r Mct^ of Hi r f erf , 
for iiiai:>|«li , is inferred fron bihavicr* • writ inj: furiuus1)r» asKinc 
questions^ yavning, fidgetinf, complaining. Fqu.illy ccnfusinf to 
th staff was the r^csje fro?* I * n i n in which pliyic.*^ nvrli'iince or 
escape from a situation could be observcc!, but requir d an additional 
atateaient specifyinc the reduction of disconfort. Finally^ no 
Increase in f ehny \ or attribute was required to shew that removal of 
the averslce stimulus was actually reinforcing. 

Hliile the overall student achiev aunt level was about as high ar. 
on any concept in the protocol unit, the liritaiions in the NR conci;»t's 
definition seemed to be sufficiently serious that one aiicht conclude 
no adequate naatery of the concept /as possible under the conditions. 

K . R . f n n . Table VllI shows that students gave a mixed report on 
the visual quality and organiration of the film with one-third ratii:g 
it below the Middle scale value and SS percent above the Midpoint. 
Sound quality was judged to be below average and was a source of 
confusion. Int ^ration of the film with written materials was 
^^ladged to he reasonably good. 

ERIC 



Firlil*tcst $taff were in d i >«i|:rccrcnt arorf. Wr- • the 

filaed fortruyMl of negative rc i n f c r cm nt . Pri tuition 
vat vhcthrr avoidance of an aver^ive atinulu* w^. otcct 
or whether, in th« caaa of the student vho wanta to go to the offiee, 
there is the pre5ence of a pos tlve reinforcer, Ine arfuneiit wn5 a 
peripheral concern and only -^cpreaent!! the dilcnna posed by the 
Motion of negative re i nf ore cment • 

Khapin;; (Sll) Students conpleting tho criteritn test on Sll tendtd 
to rake rore errors on the mk < < r ^ i vc oprrox < ri.% r i f^n attribute tb;ir 
on either of the other two u% thown in Tnble IX. Although the 
incidence of error on this concept was equal to the lowest founJ 
arong the six concepts« tlierc occurred sore confurion bctwern tlic 
attributes of Succcf r iyc Ap rroxifwat ] on and Sc Icct \ ye l^c i n f ere cr cnt . 
Part of the confusion uteris from the inconsistency in the text on 
page 107. The initial specification of attributes identifies 
Keinf on rr«cnt , "Some for* of rewarding conscquenc c • • • and 
Succes!<i yc At;prox in ut i ens , '*Re i n fore er.cnt is fiven selectively as 
the behfivlor become s aiore 1 ike. • .terminal behnvior." On pnge 1 D8 
tho labels read: Se 1 or t i vr R ein fom*: ont and Sue ce 5 ^ i v e A ppr r >: i r p t i c^n s 
and later in the text* (p. 117) the labels are chanKcd oc^in to 
Rei nf or rrnen t and Pucr os s t vc Aj>nrox < r>nt t ons with Pi f fevent i nl 
l leinforcc M.HM t in parentheses. lirrors made or the Selectiv e 
Re info rro ' ent attribute were, more often than not, due to the 
respondent's omitting the word Sol ect \ ve . 

Tho definition of fhaping presented in the text soonod to be 
more compatible with the field test staff's notion of a functional 
definition. Kith a lower orror r:ite and Hpparent definitional 
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•dcquacVf the ficld'tc«^t staff cxjrcsscd greater coufort with the 
use of this particular concept unit. 

S H, riln . I j]n quality as viewed by s* idcnts (Table ^ 
adequn t e with r.orc students report i ng f aver .ib 1 e reo c t i ons than 
unfavorable, Sound quality was poor to fail according to the 
students, but overall integration of the filr* with written text 
iras juJi;cd adequate. The field test staff also reported relative 
difficulty with sound quality, but concluded that the film was nn 
adequate reprt f^ent n t i on of shaping. 

Respo ncW ill Ten rn i n r (I^L) 'iho RL concept unit was clearly among the 
nost difficult for students. Slightly wore than half of the students 
achieved the 80 percent accuracy level, and no re than one-third of 
all student errors on the entire protoco^ pac1;agc occurred on the 
RL unit. As shown in Talilc XI, 40 percent of tho errors occurred on 
the CK attribute with the remaining four nt^-ibutes roughly equal 
sources of error. 

Rome confurion in the meaning of DCS rt because students 
believe that UCS had to be a natural or unl . inccl e\ent which 
elicited a UCR. UCS was nlao presented as .h ' :rncr state/' not 
one easily observed, e.g. p. 130, UCS. .. Hunger . It night have been 
better to describe the setting such as "absence of food for X hours," 
CR*s specification por.cd a problem similar to UCS. The so-called 
emotional or affective response was specified as the CR, but in 
most cases the feellni: had to be inferred fron the overt behavior. 
Students seeir.ed able to specify the overt beliavior, but failed to 
grasp the need to nake the additional inference of fcclinn. 

ERLC 
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Analyses of the written errors suggest to the field test staff 
that the prior, more linitod definition of respondent conditioning 
tauclit in introductory psychology classes intrrfcrcd with acquisition 
of the UCS attrihutc. An additional sou? fusion seemed to 

come from the UCR and CR attributes whic; fined as feeling 

$tatt*s inferred from behaviors which themselves were often operant 
behaviors. Students developed operant strategics to produce the 
operant behaviors from which the respondent consequences could be 
inferred. 

On page 137, the lists of U SC ' s constitute as wcil arrays of 
reinforcing stimuli. Such lists added to tlic confusion- - interference- 
betv;een the operant and respondent Icnrning concepts. In order to 
clarify the confn5;ion the field-test staff almost « vays supplcmenttJ 
the text witV, a ''Isxcturct te"' on the effective con^ uences of 
operant }iroct dur t ?; ^ Making the point tliat operant p -^ccdures always 
can be rhov;n to h.ive respondent learning attributes j resent. The 
distinction bctwc.n operant and respondent learning is larg<^ly a 
natter of procedural differences in sequencing stimuli. 

_ R . r . r i 1 m > Judged by all sources feedback, the RL film was 
the nesthcthic hit of the protocol p*cl;age. Spontaneous applause 
occurred in every class witlk smiles * rJ happy reactions among the 
students comronp 1 ace . Tnblc XIl rcvc^.s that studerr- reaction to 
visual details of tJic film was positive, ranking sec'^nd iii level 
of rating among th< six concept film: Sound qualit was judged 
to be good tt.*; was the integration of lie film with the written text. 
Field-test staff reported satisfaction with the filr and had no 
suggestions for its revision* 
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Model Learning (ML) With roughly only ten percent of all errors 
occurring on the unit and almost 80 percent of the student's 
achieving criterion accuracy level, this unit was the most successful 
of the six concept units. The two nit ^ntes most troublescrc to 
students wer* Credible Mode 1 c c r> to Mode 1 with tvo-thirds 

of all the errors made on this unit. Distribution of errors across 
attributes is shown in Table XIII. The field-test staff's rcactirn 
to the unit was uniformly favorable. Studert confusion recmed to 
be at a mi niiuum and staff generally did not have to "defend any 
examples or interpretations. 

H > L . I' i 1 m . Student reaction to the ML iilm was as poy>iti\^ it 
wns for the RL ilni. Drta in Tr.^ le XIV shov. tl-r more tlian cout 
of the St 'ents judged tne vis qualities to oe good or very K 
Audio ch..i acteri sties reccivec" nrying jud^ncnts with 23 percent 
registering disapproval while 43 percent approved of the sound ^ lity 
Integration of the film with • written text was favorably received. 
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llnc *x pcct cd C o!i'".cqucnc c s of Usinr r ^ot " ^L-^ijJ^liii,l ^ 

Several thin{;5 happened to th? U f field-test faculty ;is a 
result of their pa r t i c i t i ng in t e *SU i>rotocol field test. First, 
there developed a -h e i j^h t on cd awarencs: of the need to identify the 
key concepts in educational psych oZo^^ji:' and to structure more adequately 
the learning experience by usinp, tt.c proccdi.ics incorporated in ihe 
MSU protocols, Tlie USJ- field testers, in their ov.n classes, began 
to introduce more case examples am non- exainpl os to explicate the 
meaning of selected concepts. 

A second outcor.e v.ns to incrc . • v e]»portU!. : : i cs for student- 
pacct' ar.cl out-of'Class instructior * th .» cjncurrcr.t reduction in 
the lecture as tlic main instruct it ^ p'^ < .twc. This senncd t( 
haj^ptning before the field test, b^* 'i-e j c css aj^j^cared to be 
accelerated as a con sec; dc nee of thr irot 1 irocodure. 

Tor some the protc»rol emphasi: 4 cr.]K pis whicl) were not nor::.ally 
included in tlie beginning educatioi^-T p^\.., logy course. For ex;', j ; le, 
wodt 1 learning was not em]^hasizc d in -n.^ c f rhe clar-ser. prior to t 
use of tlie protocols, but follouin ccnclur>ion of the fiel 

test, model learning was includetl j ^hc ccurse content by all t lie 
f i 1 d testers. 

Instruction - 1 cost, a tall:cd 
by the use of ]')r. tocol iiinterials. 
nation and fec'Jl)ack rap I y cxca 
by the field-ti t faculty. The ii( 
need for severa 1 ki nds c- T i n . t rue 
need for techni a I assistance in 
evaluation of Ic tuning were aiap)> li 



1 1 ^onrc*]>t, v;as broupit to ii , 
I . |m *^ r s i n \* c I V e d j n e v a 1 - 
r (tier p r o c c u r e e ii;]-) 1 c. c u 

• • n t of special na t e r i :i ] , t ji e 
.i p :< c L for thr s n ni e class, t ;i e 

wi cnt, and tlic cost of 

♦ u in xhv experiences oi' 
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the faculty. In a sense, llic p;< r t i c i p;i l i on in the field tcM in- 
creased the sophistication of the rt i c i ]>:in t s , 

While not totally unexpected, stiulenls canie to cxpecl the sar.e 
level of or^'.an i za ti on and relevance in tliC remainder of the ecu: rse. 
There was additional j)ayaff in students ro)ortjna that they "saw a 
concept'' in a classroon visit or pre- i nternsn i p exjn^rie ce just liK 
the one descrihod in the filni or in the j^uidc. Such feedhack came 
as late as t\.'o quarters after the student was ir» the class, 

I'inally, for soiree pa r t i c i jki t i ng IISI- faculty, the contact \:\\\: 
the MSU apj^roach to educational jsycholo^^.y was a s t i ir.u 1 a t i n and 
provocative challen^^.c to as sui::pt i ons held locally. In the brc 'est 
sense, it wan an educational exju'rience for the USI faculty. 
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Note uhis for:- Jiffcis in sq-q \.ays frcr> the Mi ch i f •g^ FtnU' Ur.ivc-rrity forn . 

STU!)!:.Vr AN'ALYSIS TOUM 
Protocol Materials for Teacher Hducation 
Concept Unit Evaluation 

This forn nas been deri^^ncd to collect your ju^fr/cnt of the 1 3-i:^truct i onal 
value of the cono^pt unit you have just finished. I lers read eacli item 
carefully and be as objective ar> pc ;5iblc in rcaciiir.g your decision. 

The five nurbers follc\vir*j; each question rcprc cut a scale or continuun. 
The extremes of each >cale have been identified tc aid your ir.aliKj; this 
choice. Mark on your ansv.er sheet tlie nur.ber v.hic: best represents your 
judp.ir.cnt of t]:e degree to which the concept unit 5 ti'-.ficci ct.c1i criteric.n 
identified in the question. A concept unit inclu;' ^ both the filr.Cr) and 
the printci' r.atorials, 

PLI-ASF: DO ::X OMIT ANY ITLMS-^IL^Tr. TIIH CONChr: V' ' ' 0.\' liACH CllAPjm.lllSTlC . 

1. How clc;,r to you were the objectives of tlu co:ircpt unit? 
Anbi£Uous Clear 

1 2 3 4 5 

2. How interested vrcre you in the concept unit v;liica you have just finished? 
Bored Stiff Very Interested 

1 2 3 4 5 

3. To what extent did the concept unit build on your prcv ous know]ed^:e, 
skills, or experiences? 

No relationship Hifchly Related 

1 2 3 4 5 

4. V;as the learning cutcor? in this concept unit appropriate for your 
present understanding of teaching and learninp.? 

Not appropriate Very Appropriate 

1 2 3 4 5 

5. Did the cor.tcnt support your achicver:cnt of the instructional 
objectives of the concept unit? 

No Support Clearly Suppc--:ed 

1 2 3 4 5 

6. Was the content of the film guide presented in a well organized and 
systematic \:ay? 

Confused, Disorranizcd Vcr>' Well Orpani red 

1 2 3 4 S 

7. Kere the ixrportant idc, and concept attrib-rr- clearly r-phasi::e;' ? 
Not At All Vcrv Clear 

1 2 2 4 5 
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8. Did tl)c concept unit attempt to present too nuch rp.tcrial to be 1^ : 
at one tire? 

Very Definitely Too Much ii.tiri:^ :^: r:.,tc A*.ouni 

^234 5 

9* V/erc r.:*v; facts, ideas, terriinol oj:y or procedures introduced .'^l a r:;te 

which j c^rnitted you to Icorn then? 
Too Yii-'t Too f'lov' 

: 2 3 4 5 

10. Di^' - corcept unit ]>rovido for adcqunlo repctiticn of the inportant 
CQCJi^ ? (e.g., repetitic:-n v;itl) variation, exact rcjuniticn, sun.naries, 
our^ s . ) 

Xer^ : iTidcquate Excellent 
1 2 3 4 5 

11. Were . c iiOthods of presentation (n;otjon picture, printed materials, 
disn: .^ion) i;uit:.Mje for the subject rir.tter? 

Ver- .w:ppropriate Most Appro]>rintc 

2 3 4 5 

12. KcT-- '^0 visual dciails of the filricd sr;;nonts clrnrly presented? (Tnis 
rei:r . to cai.:era a]i[;les, lightir.^;, sliarj r.c; , cxp sure, use of closcups, 
etc. 

Vcr- VvoT Film Very Good 

1 2 3 4 5 

13. Wa.^ tl:c difficulty of the tas);s in the concept unit appropriate for 
your Ic -e] of cduc.itioiial developn.cnt? 

Toe Dif. tcult Too Easy 

1 2 3 4 5 

14. Kai the '^oand tracl: clearly audible? 

Inatdible Very Easily Heard 

1 2 3 4 5 

15. Wa: the verbal difficulty of tlic materials appropriate for you? 
To Pifficult Too Easy 

1 2 3 4 5 

16. Vic-r^ the written rnterials easy to use and to understand? 
IiET'ssihle To Use, Understaiul Very Eary To Use, Understand 

^ 2 3 4 5 

17. Ka infornatioi in the written rnterials well intc^^rated with that 
pn :t d in the hot ion picture film? 

ISo ir v;;rat ion Closely Integrated 

2 3 4 5 



TAiii.r. I 

NUMiii-R OP nur.oRS ON HACH coN'CKi'T Tnr.T r.>;rRrs5;i:!) as 

PLkCHNT or TOTAL ERkOWF, t'ADil ON' Till; SIX CONClJFT TKSTS 

rnRCfM" OP 

CONCI-I'T ERP.ORf. ALL TRWOVS 

Operant Learning 406 24.9 

Positive Rcinforcciv.ent 203 12.5 

Negative Reinforcement 128 07.8 

Shaping 128 07.8 

Respondent Learning 605 37.2 

Model Learning 158 09.7 

Totals 1628 99.9 
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TAPi.r. II 

NUMfiHR, Prr.C]:\-T of all STLIDKNTS ACIIILVTKG EIGHTY 
rCRCLNT ACLURACY ON CONCHI'T CRlTIiRION" THSTS 



CON'Cr.PT 


TOTAL 
N 


K AClllLVI.NG 
CRITHRIO:: 


PERCIiN AT 

criti;rio:> 


Operant Learning 


244 


114 


46.7 


Positive Reinforcement 


244 


174 


71.3 


Negative Reinforccinent 


244 


173 


70.9 


Shaping 


244 


1S3 


7S.0 


Respondent Learning 


24-1 


132 


54.1 


Model Learning 


244 


191 


78.3 
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TAIU-I-: in 

KUMBF.R, FLRCrNT OF ATTRirDTH ERRORS ON OPEILWT 
LEARNING CRITERION TEST 



ITHM ATTRIBUTES 





Bch.ivj or 


Consequence:; 


Temporal 


I'clat j onslu p 


Behavior 






N P 


N P 


N 


P 


N 


P 


I 2 


8 


10 


32 




6 




6 


4 


20 


30 




7 




8 


3 





30 




16 






15 3.7 


39 9.6 


92 


22.7 


29 


7.2 


II 


57 14.0 


56 13.8 


70 


17.2' 


48 


11.8 


TOTALS 


72 17.7 


95 23.4 


162 


39.9 


77 


19.0 



TM'.l .i: IV 

STUFILNT i:VALUATION' 01 OI'l-PANT COI.Dn iO.NJNG U.\'1T 

rn.M 

NUMi>r:R 

RHSP. 1 . RFSP. 2 RhsP. 3 RESI'. 4 RHSP. S T01A L MHA\' 

N P N P N P" N T N P N 

1 0 00 2 02 33 28 35 30 48 41 118 4.1 

2 2 02 10 08 44 37 38 32 35 29 119 3.8 

3 5 04 14 12 36 30 44 37 21 18 120 3.5 

4 3 02 5 04 26 22 44 37 40 34 118 3.9 

5 0 00 9 08 30 25 52 44 27 23 118 3.7 

6 2 02 7 06 36 30 41 34 34 28 ' 120 3.8 

7 1 01 9 08 19 16 47 39 43 36 119 4.0 

8 6 05 10 08 28 24 53 44 22 18 119 3.6 

9 6 05 19 16 71 61 18 16 2 02 116 2.9 

10 6 05 5 04 24 20 53 45 29 25 117 3.8 

11 3 02 7 06 28 24 48 40 33 28 119 3.8 

12 18 15 22 19 47 40 24 20 7 06 118 3.2 

13 0 00 5 04 83 72 24 21 4 03 116 3.2 

14 33 02 46 39 22 19 10 03 7 06 118 2.3 

15 0 00 7 06 84 71 27 23 0 00 118 3.2 

16 0 00 7 06 28 24 54 46 28 24 117 3.9 

17 0 00 8 07 23 20 48 41 38 32 117 3.8 
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TAUl.li V 



ERIC 



niM 



3 

s 

9 



11 



NUMl'.IiFi, Pi;RCI-.Ni OF AITKILUTM IRRORS ON I'OSITIVH 
Ri;iN'l-OU(;i MCNT CRlTLklON 7LST 



l^chavior 
?5 P 



4 
7 

5_ 
16 

23 



7.9 
11,3 



Consc(|ucT i'-('S 

N 1' 

0 

6 

2 

8 3.9 

28 13.8 



ATTRir.uir.s 

Cont i n "VDcy 
N V 

4 

15 

20 ^ 

39 19.2 

29 14.3 



'lf;!iipornl I' liiti onrhi p 
N T' 

9 

14 

29 14.3 
31 15.3 



TOTALS 31) 



19.2 



36 



17.7 



68 



33.5 



60 



29.6 
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TAlil.I- VI 



STUni:N'T liVAI.DATJO.N 01- l'031'I IVL 111 IM-ORCKMilNT U.\IT 

1T!;M 

NIIMKI-R 

luisp. 1 i'lyi:.! nijsp. 3 i ;i:.sp. a w f.w s tojak, mhan 

N P K 1' N V N 1'" N P N~ 

1 2 02 3 03 18 16 Xf. 32 51 47 109 4.2 

2 2 02 13 12 31 28 41 38 22 20 109 3.f> 

3 2 02 6 05 34 31 44 40 23 21 109 3.7 

4 4 04 2 02 27 25 38 35 38 35 109 3.9 

5 2 02 3 03 31 28 51 47 22 20 109 3.8 

6 3 03 9 08 13 12 45 41 39 36 109 4.0 

7 1 01 1 01 24 22 42 38 41 38 109 4.1 

8 5 04 7 06 34 31 43 39 20 18 109 3.0 

9 4 04 21 19 70 58 12 11 2 02 109 2.9 

10 2 02 6 05 25 23 49 45 27 25 109 3.8 

11 2 02 5 04 30 28 38 35 33 30 108 3.9 

12 10 09 21 19 32 30 32 30 13 12 108 3.1 

13 1 01 6 05 76 70 24 22 2 02 109 3.2 

14 7 06 31 29 28 26 28 26 14 13 108 3.1 

15 0 00 4 04 72 67 29 27 0 00 108 3.2 

16 0 00 3 03 25 23 47 44 32 30 107 4.0 

17 0 00 7 06 25 23 39 36 3R 35 109 4.0 
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TAIMJ. V] 1 

KUMBHR, PPRC^NT OV ATriUnUTl: TRRORS O!. :,T,CATlVr. 
RlilNrOkCrMr.NT cuitlrion tksi 









ATiuii'urr.s 








Presence 


of Pr.iTi 


Bchavi or 


Esc IPC 




N 


1' 


U P 


K 


P 


I 2 


1 




18 


2 




6 


2 




0 


1 




8 


4_ 




27 


3 






7 


5.5 


45 35 . ] 


6 


4.7 


11 


25 


]9.5 


18 1-1.1 


27 


21 .1 


TOTALS 


32 


25.0 


eS 49.2 


33 


25.8 



-2(1- 

TAi;].] 

SIlJDIiNT MIATION O!' N; .' NI-OllCKMilN-l UNIT 

III.': 

Nir-i:'! u 

RUST'. 1. I' 1'. 2 nr.sr. ? i nvsv. s totm. r'.r.AN 

K P N P K P K P K 

1 2 02 9 OS 32 28 40 36 112 3.8 

2 2 02 1?; 12 59 35 14 12 112 3.4 

3 6 05 13 12 42 37 -2 16 14 113 3.4 

4 3 03 11 10 39 34 26 30 26 113 3.6 

5 0 00 9 OR 37 33 37 25 22 113 3.7 

6 3 03 14 12 29 2(. 37 25 22 113 3.6 

7 2 02 17 15 25 22 42 21 18 113 3.6 

8 2 02 15 13 40 35 33 19 17 113 3.5 

9 7 06 24 21 65 58 12 3 03 112 2.9 

10 7 06 12 11 29 2' i> 15 14 12 112 5.5 

11 2 02 12 11 33 3 40 20 18 111 3.6 

12 14 12 :'4 21 33 2 24 14 12 112 3.0 

13 2 02 7 06 78 70 20 0 00 112 3.1 

14 15 13 37 33 41 37 .4 12 5 04 112 2.7 

15 0 00 7 06 79 70 22 20 4 04 112 3.2 

16 ' 0 00 13 12 39 35 37 33 23 20 112 3.6 

17 1 01 9 08 36 32 38 34 27 24 111 3.7 
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TAHI, !: IX 
. LN'T or ATTi:]' " T'.^l'S ON* 

CRiTL-.ijr' 
AT7: ir 





In-r ; 
Pelinv 


Success 
Appro.r i 1 


■ i CTi 


SeU t 






N 


N 




N " 


1' 


I 2 


0 


19 




3 




5 


3 


2 




1 




9 


11^ 


25 




28 






14 


46 


36. 


32 


25.0 


II 


17 i: 


10 


7.8 


9 


7.0 


TOTALS 


31 2 - 


56 


43.8 


41 


32.0 



ERIC 



STli : M 



iTi:?«t 

NUMi!!'.!'. 



3 
4 

5 
6 
7 
8 

9 
10 
11 
12 
13 
14 
15 
16 
17 



1 
A 
] 
1 
2 
1 
1 
2 
2 
2 
3 
7 
1 
7 
0 
0 
2 



I. sr. 1 

V 
01 
04 
01 
01 
02 
01 
01 
02 
02 
02 
03 
07 
01 
07 
00 
00 
02 



HF.sr 



N 
10 

9 



7 

6 

13 



10 

on 



12 12 

6 OC. 

10 10 

10 10 



07 
f 



14 14 

9 09 

18 18 

4 04 

34 34 

9 09 



17 

24 
27 
2t 
At 



5 
1 



05 
01 



24 

31 
63 

ST 
2S 



3< 



3- 



P 

^9 58 

^5 52 

15 AA 
■19 

lb 45 

38 37 

37 36 

38 37 
11 11 
44 45 
54 3 3 
23 25 

25 24 
18 18 

26 26 
40 40 
43 43 



55 34 
22 22 
17 

: .!2 



] 
2 

28 
23 
3 

18 



17 
5 



3 

24 

2- 



17 

27 

17 
22 
03 

: « 



28 27 



17 
05 



11 11 



03 
24 
24 



]!).- 

]<■ ; 

IJ.S 

10 

10 

1( 

It : 

IV.. 
10] 
102 
101 
101 
101 
99 



3 . '.I 

3. 

3.() 

3.S 

3.7 

3.R 

3.8 

3.7 

3.0 

3.() 

3.f; 

3.2 
3.3 
2.9 
3.2 
3.8 
3.8 
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TAI'.U: XI 

KUMBtR, rilRCI-NT OF ATTRirUTn TRRORS C:: RliSPONDrNT 
LnARMKG CRITKI'.JO:,' TliST 



ITBl 








ATTRir.uins 








Unconditioned 
Stimulus 


Uncondi ti onc'd 
Response 


Condi 1 3 oncd 
StinuJus 


Pairing Conditior. 




N 


V 


N I' 


N P 


N 


P N 


1 3 


20 




16 


38 


33 


37 


S 


7 




S 


17 


18 


87 


8 


39 




38 


22 


11 


76 




66 


10.9 


59 09.7 


77 12.7 


63 


10.4 200 3^ 


II 


23 


03. R 


29 04.8 


29 04.8 


19 


03.1 40 C 


TOTALS 89 


14.7 


88 14.5 


106 17.5 


82 


13.5 240 3r. 
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TA~i-i ; : I 



;';:.r.U 



1 

2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 
13 
14 
15 
16 
17 



i!i;sr. 1 



KliSP. 2 



N 
2 
4 

5 

2 
0 
4 

12 

7 

2 

1 

4 

3 

6 

2 

2 

0 



P 
02 
04 
05 
04 
02 
00 
04 

07 
02 
01 
04 
03 
06 
02 
02 
00 



N 
9 



P 
09 



11 11 
11 11 



7 
9 



9 

13 
S 

15 
9 



07 
09 
05 



11 11 

9 09 
22 22 
16 16 
7 07 

10 IC 



09 

12 

m 



2t, 
3" 
3." 
30 
40 
29 
27 
40 
57 
3] 
31 
37 
66 
34 
68 
40 
27 



' . 3 
P 

25 

32 

32 

29 

39 

28 

26 

39 

56 

50 

30 

56 

64 

33 

67 

39 

27 



32 
36 
37 
44 
35 
31 
33 
27 
13 
42 
39 
21 
22 
23 



I' 
31 
35 
56 
43 
34 
30 
32 
26 
13 
41 
38 
21 
21 
22 
25 25 
28 27 
38 58 



;-.7 

27 

]4 

3 

12 

25 

30 

3 

26 
1 

1" 

21 



it 

\i 

17 

]6 

3ti 

2io 

14 

03 

12 

24 

29 

03 

25 

ai 

17 
27 



K 

102 
103 
103 
103 
103 
102 
102 
102 
102 
103 
103 
102 
103 
102 
101 
102 
101 



.■> . . 

3 • 

3 . r. 

5.1' 

3 . 

. 0 
3.7 
3.2 
J.S 
3.4 
3.> 
3 . 

3. : 
3, 
3.-: 
3 . • - 
3.f 
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r/vBLR, '!■.(■'.. or Am]!:UTi- ru: ••odi-.l 
MnnnL Kc. ' Monui. ,::i '\vior _ 

N I' N 1' K I' ,\' 1^ K ' i- 

15] 13 11 2 

5 2 4 1 1 2 

8 £ 21_ 6 ]_5 4 

8 5.1 38 24.1 8 2 17 10.8 8 .0 

II 44 27. S It '0.1 7 4_ 7 - ^ ' 

32.9 54 34.2 15 » 24 1 ' .2 1? .2 
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TAKl.r. vi\ 



s'" :ini,:;r lvaij vi i;.:; ov 



ir.iT 



IThM 

1 

2 
3 
4 
5 
C 
7 
8 
9 
10 
11 
12 
13 
14 
15 
16 
17 



1 
3 
1 

2 
1 
2 
J 
3 
6 
1 
3 
6 
1 
6 
0 
1 
3 



01 

05 
01 

02 
01 

02 

01 
03 
06 

01 
03 
06 
01 
06 
00 
01 
03 



6 

10 

7 

0 

6 

10 

10 



03 
Of. 
10 
07 
CO 
06 
]0 
OS 
]« 
06 
Oc 

04 
17 
06 
05 
06 



RrS!'. 3 

21 20 
25 24 
37 3f. 
29 28 
31 30 
15 14 

22 21 
33 32 
6J 66 



26 



21 
1 ■■ 



60 

54 3 
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